Estimating the Maximum Expected Value: An Analysis of (Nested) Cross Validation and the Maximum Sample Average

We investigate the accuracy of the two most common estimators for the maximum expected value of a general set of random variables: a generalization of the maximum sample average, and cross validation. No unbiased estimator exists and we show that it is non-trivial to select a good estimator without knowledge about the distributions of the random variables. We investigate and bound the bias and variance of the aforementioned estimators and prove consistency. The variance of cross validation can be significantly reduced, but not without risking a large bias. The bias and variance of different variants of cross validation are shown to be very problem-dependent, and a wrong choice can lead to very inaccurate estimates

Bayesian Sampling Algorithms for the Sample Selection and Two-Part Models

This paper considers two models to deal with an outcome variable that contains a large fraction of zeros, such as individual expenditures on health care: a sample-selection model and a two-part model. The sample-selection model uses two possibly correlated processes to determine the outcome: a decision process and an outcome process; conditional on a favorable decision, the outcome is observed. The two-part model comprises uncorrelated decision and outcome processes. The paper addresses the issue of selecting between these two models. With a Gaussian specification of the likelihood, the models are nested and inference can focus on the correlation coefficient. Using a fully parametric Bayesian approach, I present sampling algorithms for the model parameters that are based on data augmentation. In addition to the sampler output of the correlation coefficient, a Bayes factor can be computed to distinguish between the models. The paper illustrates the methods and their potential pitfalls using simulated data setsSample Selection, Data Augmentation, Gibbs Sampling

Deep Reinforcement Learning with Double Q-learning

The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.Comment: AAAI 201

Individual differences in maternal care as a predictor for phenotypic variation later in life

Vroege levenservaringen hebben een grote invloed op hoe we later in het leven functioneren. Studies hebben aangetoond dat verstoringen van de leefomgeving in de jeugd de kans op het ontwikkelen van allerlei aandoeningen in volwassenheid vergroten. Met behulp van een model met ratten, waarbij wordt uitgegaan van een natuurlijke variatie in de hoeveelheid moederzorg die optreedt in een populatie, onderzocht Felisa van Hasselt wat het effect is van verschillen in ontvangen moederzorg op de karakteristieken van een dier later in zijn leven. De hoeveelheid moederzorg verschilt niet alleen sterk tussen verschillende nesten, maar vertoont ook een behoorlijke variatie tussen individuele pups binnen elk nest. Deze individuele verschillen in moederzorg correleren direct met structurele en functionele parameters in de volwassen hippocampus, een hersengebied dat betrokken is bij leren en geheugen en dat erg gevoelig is voor stress. Ook de expressie van bepaalde genen die bij deze processen betrokken zijn lijkt gerelateerd te zijn aan de hoeveelheid ontvangen moederzorg. Ten slotte bleek zowel speelgedrag als keuzegedrag, maar nauwelijks de prestatie in hippocampus-afhankelijke leertaken, samen te hangen met de hoeveelheid moederzorg die een dier vlak na de geboorte ontving. Voor sommige van bovenstaande parameters bleken de effecten op mannetjes en vrouwtjes verschillend

Effect of linear polarisability and local fields on surface SHG

A discrete dipole model has been developed to describe Surface Second Harmonic Generation by centrosymmetric semiconductors. The double cell method, which enables the linear reflection problem to be solved numerically for semi-infinite systems, has been extended for the nonlinear case. It is shown that a single layer of nonlinear electric dipoles at the surface and nonlocal effects allows to describe the angle of incidence dependent anisotropic SHG obtained from oxidised Si(001) wafers. The influence of the linear response, turns out to be essential to understand the anisotropic SHG-process

What doctors should look for in patients presenting with erectile dysfunction

Click on the link to view the commentary.S Afr Psychiatry Rev 2003;6:29-3